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Work  on  tliis  tliesis  was  begun  durine  the  first  elerven  weeks  of  IS^k 
wiiile  the  writer  was  er^^loyed  try  Uie   electronics  design  and  aanufac tun- 
ing fim  of  Gilfillan  Brothers,  Incorporated,  located  at  Los  Angeles, 
California.  This  enployment  was  the  industrial  experience  tour  in  the 
aigineering  Electronics  Curriculum  of  the  United  States  llaval  postgrad- 
uate School  wtiich  requires  tiie  student  to  work  as  a  junior  engineer  in 
the  electronics  coxaparor  of  his  choice.  Coii5)letion  of  the  thesis  was 
accon^Dlished  during  the  Spring  of  19$U   at  the  United  States  llaval  Post- 
graduate School. 

This  investigation  was  done  in  cooperation  Tilth  iir.  Louis  A.  Ule, 
a  meuber  of  tlie  Theoretical  Analysis  Group  in  the  £i:igineering  Department 
of  Gilfillan  Brothers,  Incorporated.  Its  purpose  was  to  round  out  the 
least  squares  siaDotliing  and  prediction  theory  embodied  in  Mr.  Ule's 
intra-coapany  oeiaarandun  of  Januaiy  12,  19f?l4,  entitled,  "Polynonial 
Filters."  "While  a  considerable  amount  of  material  in  the  thesis  is  tlie 
result  of  joint  discussion  or  discovery,  the  fonaulation  of  the  weighting 
function  (Ciiapter  II,  Section  h)   and  the  proof  of  the  steac^  state  filter 
re^jonse  in  the  absence  of  noise  (Chapter  II,  Section  5)  are  abstracts 
of  material  vrritten  by  Ur»   Ule. 

The  writer  wishes  to  e:xpress  his  appreciation  to  Mr.  Ule  for  his 
guidance,  assistance,  and  original  material.  He  is  also  indebted  to 
Professors  Abraham  Sheingold  and  Robert  Kahal  of  the  United  iitates  Naval 
Postgraduate  School  Staff  for  theii  suggestions  anu.  encourageneno  while 
acting  as  thesis  advisers  and  readers. 
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SYIIOPSIS 

The  smoothing  and  prediction  problem  is  one  of  oxtractinc  fron  a 
noi^  sienal  tlie  best  estimate  of  some  functional  of  the  true  signal  at 
some  past  present,  or  future  time.  The  historical  development  of  the 
least  sqiiares  solution  to  the  problem  is  briefly  discussed  alonp  vrLth 
other  backcround  information  relating  to  this  process. 

A  method  of  smoothing  and  prediction  is  developed  which  is  a  (^/nanic 
fbKn  of  least  squares  curve  fitting.  Tlie  true  signal  is  assumed  to'  be 
conposed  of  completely  forelmoi-oi  functions  of  time  each  multiplied  by  a 
constant  coefficient.  The  filter  v;eigl:ting  function  is  chosen  so  that 
the  filter  output  will  be  cojiposed  of  the  same  functions  as  the  true 
signal  e:a:ept  for  a  possible  translation  in  time,  differentiation,  or 
integration.  The  squared  error  is  tir.ie  vreightod  before  being  minimised. 
Yfliile  the  derivation  of  tiie  -weighting  function  is  independent  of  the  tyi)e 
of  noise,  the  choice  of  the  squared  error  vfeighting  fLuiction  may  depend 
upon  the  character  of  the  noise.  Rmctions  wliich  car.  be  treated  are  found 
to  be  the  solutions  of  the  reduced  equation  of  a  linear  differential 
equation  vdth  constant  coefficients.  Tlie  resultant  filter  vreighting 
function  depends  upon  the  true  signal,  the  prediction  tL~e,  the  squared 
error  vreighting,  and  an  inverse  matrix  vriiich  is  the  solution  of  a  set  of 
int^ral  equations. 

Tlie  resultant  filters  are  constructed  of  linear  elements  T,iiich  do  not 
vaiy  Tdth  tine.  Although  no  conditions  for  plxysical  realisability  ai^ 
ix^xjscd,  a  high  probability  exists  that  the  filters  may  be  constructed  by 
one  of  several  methods.  A  step  hy   step  procedure  for  the  filter  design  is 
given  together  tdth  specific  ezsamples  of  its  use.  A  phenomenon  al:in  to 
resonance  is  observed  for  e:q?onential  and  pol;>'7iomial  fimctions. 


CLlAFim  I     UnBODUCTIUII  Tu  tile  SLlOOTiilliG  A1;D  PK£DICTlUli  PiiOBLai 

1,     Description  o£  the  Problecu 

The  aoDotliiiig  prohlen  is  one  oi  c:d:.racting  true  signals  fron  noisy 

2 
ones»     This  problen  lias  been  doscribed  tiy  Bodo  and  Sliannon    and  by  Zadeh 

and  llaca^aini-^^.     It  is  usually  acsuLiod  tiiat  a  civen  perturbed  signal, 

R(t),   is  tlie  sun  oi  a  true  signaJ.,   P(t),   and  a  i-andon  disturbance  or 

noise,  N(t)» 

R(t)    =     P(t)    +    N(t)  (1) 

^  perfonaing  appropriate  operations  on  R(t),   it  is  desired  to  obtain  a 
result  Q(t)  vdiicli  is  tlie  best  appro^dj^ation  of  P(t)«     Tliis  sraoothing 
process  oay  be  corjbir.ed  vrith  prediction  to  provide  a  continuous  appro:*- 
imtion  of  P(t)  at  sone  pact  or  future  tioc.     In  other  vrords,  given  R(t) 
and  a  fixed  preaictioa  tine  ©,  -vvliat  is  P(t  f  9)?     6  nay  be  positive, 
negative,  or  zero  dei>ending  upon  vriiether  prediction,   tiue  delay,  or 
suootiii::g  alone  is  desired.     Both  six>otliing  and  prediction  nay  be  fuj>- 
ther  coDbined  "with  differentiation  and  integration  so  that  in  the  gei>- 
eral  case  sone  functional  of  P(t  •♦•  6)  is  obtained.     Coraxmly  desirod 
functionals  are  of  tlie  fom  P(t  +  Q),  dP(t  +  e)/dt  and  /  P(t  4-  9)  dt« 
The  problen  of  filling  the  box  mariced  "?"  is  illustrated  in  Figure  1« 


R(t)  =  P(t)  +  ll(t) 


nOptinum" 
Transfer 
Operator 
? 


v^(t)  =  best  ^projdnation 
of  functional  of  P(t  +  9) 


Figure  1.  Tlie  Smoothing  and  Prediction  Problen. 
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As  a  class,   tiie  pliCr'sical  covuitcr^-arts  of  tho  "optliJLua"  traiiGi'er 
operators  ai'o  co^xmly  iaiO\-.ii  as  a^jootiixng  and  prodictinc  i'iltci's,    esti- 
mators,  or  predictors.     Since  an^r  one  filtei"  ua^  ijot  perfon.i  all  of  tlie 
aforesaid  processes,    spccixic  naues  to  ii;dicate  cz-jj^tl^  v/lut  cac.^  filter 
docs  i.iLGat  be  in  oixier.     ii-xuqjles  of  such  na:.ies  vfould  be  "reproducer"  to 
indicate  a  siuootliinc  process  vrLth  no  time  la^;  or  lead;     "differentiatinc 
predictor"  to  iiidicate  both  differentiation  and  sr-jootiiinc  vri-tli  a  fixed 
tiiiie  lead;     and  "inte^jratlnc  dela(;,^cr"  to  indicate  botii  intecration  and 
srjootitLnc  '^''ith  a  fi:axi  time  delajy-» 

The  design  of  an  optinum  Tiredictor  involves  t-.vo  questions.     In  the 
first  place,  vfliat  ai'e  ti.e  transfer  ciiaractciUGtics  for  tiie  "0]jtiiiiuia" 
ti^aiisfur  operatoi"?     Secondly,   hcni  naj.'  tiiese  c}:ai*actcri sties  be  pl'^'sicaLly 
realized?     Before  the  first  question  can  be  ansircred,    a  criterion  of 
Xjerforuance  aust  be  chosen  and  a  nethod  of  cliaracteriziin,^  the  signal  and 
noise  mist  be  fouid.     I^*  choosing  a  realistic  criterion,   the  ansi-rer  to 
both  questions  nay  be  greatl;/'  siiic^lified.     Tlie  proper  characterization 
of  tlie  signal  and  noise  will  peruit  one  to  nal:e  the  optinun  use  of  the 
a  ijriori  inforuation  conccmii:g  the  signal  and  noise. 

"tihile  the  sooothing  and  prediction  projlei.i  is  usuaUy  thought  of  as 
one  in  coiai^nmication  tiieor^^,   it  mght  vrell  be  generalized  to  cover  sux;h 
diverse  fields  as  gunner^. ,   econoiaic  prediction,  vreather  forecastii^^  and 
statistics. 

2.     Perfomance  Criteria. 

To  attaci:  the  probleu  Datheiuaticaliy,  a  criterion  of  pcrforaance  or 
laeasure  of  fidelity  :.iust  be  cliosen  for  the  optinun  pi*edictor.     Hiis 
criterion  sho-uid  be  realistic  iii  order  tliat  tiie  results  liave  meaning. 


It  sliould  also  be  sJuif^le  in  order  tliat  the  natlicj.Tatical  treatment  may  be 
economically  iX)ssible.     liornaljly  negative  errors  are  ao  undesirable  aa 
positive  ones  so  tliat  tlie  chosen  criterion  sliould  be  an  even  function  of 
error. 

o 

GralioLi  and  Latlirop     iiave  e:zLinincd  tlie  criteria  for  tiie  "optiuun" 
transient  resi-oiise  in  linear  transfer  systcias  called  "duplicators,"  in 
wliich  tlie  output  approidaateJir  duplicates  tlie  ini^ut.     Thcr,'  have  concluded 
that  tlie  mi  ni.rauri  int^ral  of  tiie  titio-mLLtiplied  absolute  value  of  error 


criterion  is  best. 

(2) 


I   =     /      t    e(t;     dt   =    niniimn 


o 

The  results  of  tiiis  article  do  not  appear  to  meet  tiic  requirements  of 
the  soDotliing  and  i^rcdiction  problem  since  tlie  lack  of  a  noise  cor^jonent 
in  the  ii^Dut  signal  has  been  tacitily  assumed. 

For  signals  vfliich  consist  of  a  numoer  of  diacrete  pieces  of  data, 
the  jjrobabili.ty  tiiat  an  error  eid-sts  might  oe  used  as  a  measure  of 
fidelity.     It  is  Lieaninglcss  in  the  case  of  continuous  signals  in  the 
presence  of  noise  since  the  probability  that  an  crix^r  TdU  occur  at  any 
given  point  is  alivays  one. 

If  tlie  error  is  designated  as  e(t)  and  the  mean  value  of  a  quantity 
by  a  bar  above  tiiat  quantity,   a  pcrforLiance  criterion  for  continuous 
signals  might  e^loy  the  minimisation  of  one  of  the  folloiTLng: 


'(t)| 


e     (t)  Tflicre  n  is  a  positive  iiit^er 


E 


a    e^(t) 
n 


n 


probability  tliat   |e(t)    '^  a  si  ccificd  lir.iit 

probabiJLit,^-  that  uission  is  not  accociplialied 

The  ocan-sqiiare  error  ciTltorion  introduced  tr/  Kliarkevitch      lias 

found  yn.(j.c  uaoi^e*     Tids  critci'ion  requires  tliat: 
To 

e2(t)   =    — i^ —   /      e^(t)  dt    =   uininun  (3) 


Frou  equation  (3)   it  is  evident  tliat  the  undcsirability  of  an  error  is 
vreichted  accordinc  to  the  square  of  its  ctacnitude.     Hie  principle  attGi>- 
tion  is  i^aid  to  nakinc  the  vciy  lai^c©  errors  as  Siia'il  as  possible  ivith 
little  recard  to  soall  errors,     Vfitliin  tlie  tioe  rantje  of  interest  tlie 
T/eiGhting  is  independent  of  the  tiiae  v/hon  tlie  error  occurs. 

Tliere  are  several  reasons  iTiy  this  criterion  has  found  such  Tri.de 
acceptance.     First  of  aH,   it  is  generally  applicable  and  in  reasonable 
accord  Yri-tli  pliysical  requirements.     Ilunerous  cases  e:ast  in  vriiich  tlie 
undcsirability  of  an  error  increases  mtli  its  ija^^nitude.     ISie  i:eai>-9quare 
error  criterion  is  reasonable  vrfien  tiie  error  lias  a  nomal  or  Gaussian 
distribution  y/liich  is  independcjnt  of  tine,  a  condition  ofter  encountered 
in  practice.     In  the  second  place,   the  nean-squai^e  error  oetliod  elirri.- 
nates  nucli  of  the  riatheiiiatical  coi:42le:d.ty  involved  in  the  use  of  other 
criteria.     A  liighly  developed  body  of  laathciaatical  Imorrlcdce  lias  oeen 
built  around  the  idea  of  the  aean-squai^  value.     In  soiae  cases,   it  is  tlie 
only  criterion  \7liich  is  aoenablc  to  Liatheuatical  treatucnt.     Finally, 
tills  criterion  leads  to  useful  results.     InfbrLiation  can  often  be  ol>- 
tained  fron  tids  uetiiod  even  vriien  it  is  not  strictly  a.^plicable. 

The  oeai-h-square  criterion  does  not  alv/ays  neet  the  requirenents  of 
tile  systen.     Kagazaini  and  2adeii      iiave  presented  the  case  in  Tjhich  all 


errors  larccr  tiuni  a  certain  limit  are  ocjual  l^-  oad,  v/ldlc  those  less 
tiian  the  linit  are  ocpially  acceptable.     Tliic  ia  the  claasic  ijuniieiy 
problea  best  dcscriued  try  the  a^dLom,   "A  miss  is  as  cc>c>d  as  a  niile." 
Cases  nny  occiir  in  vriiich  tiie  iiixiesirability  of  an  error  aci^cnds  upon 
tile  tiue  at  -ivhicli  the  error  occurs.     For  ciwauple,   tiie  undesirability 
or  an  error  dviriiig  a  transition  fron  one  inode  of  operation  to  anotlior 
j3Sif  be  quite  different  frora  that  vrliile  operating  in  a  given  i^ode, 
Ano tiler  case  in  vriiicli  tixe  iiieai>- square  error  criterion  vrould  not  appQy 
involves  tiie  tines  v;iien  as  uai:^  veiy  accurate  preuictions  as  ixissible 
are  desired  and  an  occasional  gross  cn'or  is  of  little  consequence. 

3»     Historical  Developnent, 

Uie  Urst  successful  atteript  in  tliis  countiy  to  put  tlie  noise 
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probleiu  on  a  iiathemtical  basis  v;as  Ilorbcrt  Wiener' s^  classic  report 

for  tiic  National  Defense  Research  Council  entitled,   "Tiic  Eztrapolation, 
InteriX)lation,  and  Snootliing  of  Stationaiy  Tine  Serilcs, "  which  appeared 
in  February'-  19h2»     An  earlier,    simJU'ir  but  not  identical  article  by 
A«  !]•  Koloogoroff^  appeared  in  a  liussian  scientifJc  journal  in  l^Ul* 
Altiiougii  both  Trorlis  are  devoted  to  a  study  of  tiie  optimum  prediction 
problen  tiirough  tiie  use  of  statistical  netliods  and  support  each  otlicr, 
tiiey  are  tlie  result  of  independent  investigations. 

The  Wienc3>Koli"iogoroff  tlieory  is  based  on  tlirco  assuiptions  T;hich 
limit  its  I'ange  of  application. 

1*  Hie  signal  and  noise  uay  be  satisfactorily  represented  br,'  stiitionaiy 
time  series,  tiiat  is,  tlie  statistical  properties  of  the  signal  and  noise 
do  not  ciiange  idtii  tioe. 


2.  Tile  irrininiin  oean-square  error  is  a  reasonable  cirLtcrion  of  perform- 
ance. Tlie  onsQiable  averace  is  tal:cn  over  all  iosaiblc  ci^jnal  and  noise 
runctions,   each  v^ei(;iited  accordiii£  to  its  probability  of  occurajice. 

3.  Hie  operation  used  for  sooothinc  and  prediction  is  linear  so  that 
it  a^  be  porforiiied  by  a  ..hysicnlly  realisable  filter. 

For  tliese  rea:;ons  the  theory  has  been  described  as  "linear  least  square 
prediction  and  soootliinG  of  stationary  tine  series."     If  a  civen  tine 
scries  is  stationaiy  and  the  correlation  functions  of  tlie  signal  and 
noise  are  laiomi,  tliis  tiieoiy  nay  be  used  to  obtain  tlie  specifications 

for  an  optirmij  predictor. 

23 
An  early  application  of  Wiener's  vrork  ty  Pliillips  and  V»eiss 

eartended  tlie  tlieoi^'  to  quasi- stationary  processes,  tliat  is,  processes 
wliich  are  stationary/  in  the  ranee  of  interest.     The  probleii  under 
investication  vas  the  sriootliinc  of  data  obtained  in  the  tracldjiG  of  a 
gUTJieij''  target.     In  this  case,   tlie  signal  and  noise  characteristics 
are  considered  invariant  oxept  vrhen  the  tai^et  cliangcs  course.     Then 
the  error  is  large  enough  so  that  the  possibility  of  a  lit  is  negli- 
gible and  the  results  need  not  be  considered. 

Because  of  tlie  for-iidable  laathenatics  involved  in  Wiener's  theory, 

21  22 
Ifonnan  Levinson     '       has  \/ritten  trfo  papers  v/ith  a  vie;7  to^rard  facili- 
tating tlie  couputational  procedure  and  siuicxirising  tlie  significant 

2 

results.     Bode  and  Shannon    have  presented  a  sin^Dlified  derivation  of 

the  theory  vdiich  does  not  resort  to  the  use  of  integral  ci-iuations  or 

auto-correlation  functions.     Recent  books  on  infoniation  theory,   such 

7  1 

as  tliose  by  GolxLjan    and  Bell  ,  discuss  tlie  noise  problem. 

30 
Zadeh  and  Eagazaini      have  furtlier  eirbended  tiie  theory  in  two  Trays. 

The  desired  signal  Day  also  contain  a  non-random  function  of  tiLie  vfliicli 


is  reproseiTtable  as  a  ]-)o]l;yTiornial  of  decree  not  greater  tlian  n  and  about 
which  no  infox'aation  otlicr  tlian  n  is  laiovm.  In  addition,  the  icQmlcivc 
response  of  tlie  predictor  uay  vanisli  outside  a  s^iecified  tirae  interval 

0  ^  t  ^  T,  vfheroas  T  is  infinite  in  Wiener' s  thooiy. 

12 
A  recent  i^aper  Toy  Hauser      discusses  tiie  geonctric  aspects  of  the 

least  squares  sLTootliing  process. 

h*     Tine  versus  Frequency  Donain  Linear  Filters. 

Tlie  nost  iL^xirtant  prerequisite  of  aro''  filter  is  tliat  it  be  pl:^sif- 
cally  realisable.     Unless  tliis  condition  is  net,  any  "optinun"  transfer 
operation  is  of  acadeiiic  interest  only.     For  ease  of  nathenatical 
ti'eatiaeiit  euid  practical  realizability,   it  is  usually  desirable  that  tlie 
filter  also  be  fixed  and  linear.     Jac;cs  and  Weiss       l:iave  indicated  tlie 
cliaroctcristics  of  the  nomal  resiDonse  of  a  linear  filter.     Tiie  nomal 
resi-onse  should  bo  a  linear  function  of  tlie  input  so  tiiat  the  linear 
superi^osition  of  inputs  and  resultant  reqDonses  vrill  hold.     It  sliould 
depend  only  on  the  past  and  present  values  of  the  input.     In  addition, 
tliis  res2X>nse  should  be  independent  of  the  tine  origin.     This  means 
that  the  values  of  circuit  elecients  are  fixed  and  independent  of  tirae. 

The  classical  approach  to  the  problem  of  separating  signals  from 
interference,  vdietlier  due  to  noise  or  other  signals,  is  to  provide  a 
filter  of  liixLted  bandva.dth  as  determined  by  tlie  stca<^  state  fi-ecuency 
components  of  tlie  desired  signal.     Ilie  resonant  tuned  circuit  for  sinu- 
soidaJL  signals  is  a  good  eicair^jle  of  tiiis  teclini»:iue.     For  signals  of  a 
transient  nature  Avtiich  liave  neither  bandiridtli  nor  frequency  con^xsnents 
in  the  usual  sense,  tiiis  iiiethod  ma;;,^  be  clum^. 

With  the  increased  use  of  tlie  operational  mathematics,   it  lias  become 
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conventional  to  utilize  the  Laplace  trancfom  v.-iicn  dcolinc  rdth  filters 
for  transient  sicnals*  !3ie  situation  is  represented  \rj   I'icure  2» 


F(s)   *-       H(s)   >-       G(s) 


Figure  2.     Frequency  Donain  Filtering. 

F(s)  and  G(s)  are  tlie  Laplace  transfoiTis  of  tlie  input  and  output 
signals  respectively.     Hie  transfer  function  of  the  filter  H(3)  is 
defined  as  tiie  ratio  of  the  Laplace  transfo ms  of  aiy  nomal  response 
and  tiie  input  tl^at  produces  it.     It  foUoars  tiiat 

G(s)   =    F(s)  H(s)  (U) 

The  filtering  of  signals  v.iiich  are  functions  of  time  can  also  be 
done  directly'  in  tlie  tiue  dooain.     The  input  sigiial  is  f  (t)  and  tlie 
filter  is  coiq.letely  described  by  its  i/eightijig  fujxtior^W(t).     T^iis 
iTeighting  function,   often  called  the  itipulsive  rcs:x)nse  of  tne  filter, 
is  the  nomal  response  of  tiie  filter  to  a  unit  i..pulse  applied  at  zero 
tiiiie.     The  'acighting  function  ezq'^i'esGes  tiie  c:rtent  to  T.idch  the  distant 
past  of  tiie  iiiput  affects  tiie  present  filter  rcs..onse  and  hence  Day  be 
tiiought  of  as  the  oeijoiy  of  the  filter.     Tiie  output  of  the  filter,  g(t), 
is  given  \yj  tiie  convolution  or  suiocrp-osition  integral  in  tcrr.x  of  f  (t)> 
W(t),   and  a  duLi-y  vaiiable  of  integration  C. 

g(t)  =  f(t)  *w(t)  =   /    f(t-r)  w(r)  dv  (5) 

Tiiae  doriain  filtering  is  illustrated  in  Fignire  3« 


i:(t)     >      w(t)      ►     c(t) 


Fi^iire  3»     Tine  Domain  Filtering. 

Hie  follcn-djiG  corresiX)nd.ence  e:d.sts  between  tlie  tine  and  fl'equency 
donain  diiltcrs. 

F(s)  =Z  [ret)]  (6) 

H(s)  =.2!  [w(t)]  (7) 

Tliis  duality  betvfoen  tlie  tine  and  frequency  donains  is  further  shorni  "by 
tiie  Gardner  and  Barnes     canplex  nultiplication  Laplace  transforji  pairs. 

£^it)  *  f^(t)   -*-^  F^(s)  F^Cs) 

A  teclmiquc  utilisiixj  both  tlie  tine  and  fre  ^uency  domains  vd.ll  be 
used  later.     It  consists  of  findiiic  tlie  appropriate  vreightiiTe  function^ 
■W(t),   and  tlien  talduic  its  Laplace  transfona  to  obtain  Il(s).     In  tliis 
case  tlie  properties  of  tlie  filter  are  i^vc  readily-  found  in  the  time 
dorjain.     Tiie  srritch  to  the  frequency  donain  is  eaployed  so  that  coi>- 
ventional  methods  of  nctvrorl:  sjTithcsis  nay  be  used. 

5.     lluncrical  Least  Squares  CXirve  RLttinc. 

Related  to  the  discussion  of  the  second  section  of  tiiis  cliapter  is 

26 

the  static  case  of  least  sqiiares  curve  fitting.     Scarboroijgli      lias  dis- 
cussed botli  the  basis  for  the  uetliod  and  tlie  steps  necessaiy  to  cany 
out  tile  nuncrical  coi4:)Utation.     It  is  based  on  the  Ilonjal  Lasr  of  Error, 
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tile  Principle  oT  Leact  Squai^cs,   aiid  tlic  Lav/  or  Error  Tor  Itesidiials. 
Consider  uiie  ruiiction,   f,  vndch  is  tiie  sum  ox  several  rimctions,   f,  (x), 

r^Cx),  .  .  .  yx). 


P  =^    b^  i^(x)     4    b^  f.^Cx)     +     .   .    .     +    b^  £^{x)  (8) 

Pairs  of  observed  values,    (x^,   P,),    (x^*  Pp)^   •   •    •    (^  >   ^  )»   ^xe  ob- 

taiiiea  iron  a  set  of  m  raeasui'eocnts  oi'  equal  precision.     It  is  desired 
to  find  the  values  oi  the  coefiicicnts,  b,,   b^,    .    .    .  b  ,    sucli  tiiat  a 
G^'a^jh  Ox  P  Trill  corje  as  near  as  possiole  to  each  oi  tiie  u  points.     A 
set  Oi  u  residual  oquatior^  is  obtaxned  by  substitutiiic  each  of  tiic 
pairs  of  values  in  turn  into  equation  (8). 

^1   "^    ^A(Xi)     •♦•     ^2^2^^^     +     .   .   .     f     b^f^^(x^)     -     P^  {9) 

^2    "^    ^^^^-2^     ^     ^2^2^^^     4-     .   .    .     f     Vn^^^     "     ^2  ^^^^ 


^m   ^   \h^\)     +     ^2^\(^)     ^     .    .    .     +     b^f^(yg     -     P^  (n) 

Hie  best  values  of  the  uxilaio\.Ti  coefficients  are  -Uiose  vfiiicli  laake 
tlie  suQ  of  tne  squares  of  tiie  residuals  a  uiniimicu 

v^*"    f     v^      i     ...«*■     v^      «    f(b^,  b^j    .   .   .  b^)    =    ni.niniu.i       (12) 

To  iaai:e  equation  (12)  a  liaidjoun  or  iiininuni,   the  partial  derivatives  of 

f(b,,  b^,   .    .   .  b  )  vritii  respect  to  each  of  tiie  coefficients  siiall  be 

1      2  n 

aero.     Tlie  vnluec  of  tlie  residuals,    v.,,   v^,    .    .    .   v  ,  given  in  equations 
(9)  tlirougii  (11)  are  substituted  into  equation  (12)  j  the  partial  differ^ 


n 


eiitiations  are  i.orron.iod  vritii  re-j  cct  to  b.,   b  j    .    .    .  b  ;   and  tiie  n 

1   4.        n 

resultinc  equations  are  equated  to  zero.  After  dividiiic  tiiTouch  bry  2# 
a  set  o£   sirailtaneouc  c^juations,  called  norual  equations,  are  obtained. 
They  inoy  be  solved  by  ordinaiy  algebra. 

f^(x^)[b^f^(x^)  +  b^r^(.^)  ,...,  b^^r^c^^)  -  P^] 


1     n 


b  f  (xj  +  b,  f  (xj  f  .    .   .  f  b  f  (x  )  -  P^ 

_1  1     J.  ^  I     d  unL:  * 

b  f  (x  )  +  b  f^Cx  )  f  .   .   .  f  b  f  (x  )  -  p' 

lln  d  c.     11  nnn  n 


■I-  .    .    . 


f.(x^)[b^r^(x^)  +  b.f^(x^)  +  .  .  .  +  b^r^(.:^)  -  P^] 


b  f  (x,,)  +  h  i.Xx.^)  ■(....+  b  r  (x  )  -  f, 

1    X       I.              dec                                    Ti  T\      d  d 

b  1    (x  )  +  b  f   (x  )  +   .    .    .   4-  b  f  (x  )  -  P 

lln           2  2     n                           n  n    la  n 


•f  .    .    . 

=     0 


(13) 


(11;) 


fb^r^Cx^)  .  b^i^(x^)  .  .  .  .  ^  b^r^(x^)  .  P,^ 


n     t! 

•I-  i:  (-^  ) 

n    u 


b  r  (x  ;  +  b  f  (x  )  +  ...  +  bf(x)-P 

1  1    u  2  2    a  n  n    n  n 


■♦•... 

=      0 


(35) 


Ij?  each  unloioa-.Ti  coefficient  is  to  be  uniquely  detcmiinod,  the 
nuobcr  of  indci-'cndent  sets  of  observations  Loist  be  equal  to  or  (jre^iter 
tlian  tiie  nui:ii:>er  of  coefficients.     I^  increasing  tlie  number  of  obsc2>- 
vations,   a  better  fit  is  obtained* 

"rilien  ueasurcLients  ai'e  of  unequal  ;jrecisicn,   the  least  square  netliod 
is  extended  by  liaviii;^  the  siin  of  the  Treichtod  squares  of  tiie  residuals  a 


LDjnxun* 


E 

a*i 


2  2  2  2 

^j'^^j      ^    ^1^1      "*"    ^^2'^2      "♦■•••+    w^Vj^      =    nininun         (l6) 
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Tlie  TTciclits,  Tr-j_,  vf2*   .    •    .  Wj^,  depend  upon  the  rrccision  irith  T/iiich  each 
pair  of  ncacurencnts  is  laade,   tlie  norc  precise  neacurcnantc  rcceivinc 
creator  v^eichtc. 


6»     Tlie  Finite  Fourier  Scries  as  a  Jifijiinuin  .'.ican-Square  Appro: dxiation* 

As  an  ijitroduction  to  a  teclmiciuc  Avhich  t.III  be  utilised  later, 
the  least  square  propcrtr^'-  of  the  finite  Iburicr  series  vdll  be  slio^m. 
Let  a  function,   f(t),  vrhich  is  piecevri.se  continuous  and  defined  from 
c  ^   t  ^  c     +     277",  be  appro:zLnatcd  by  the  fimction  G(t)» 

c(t)    =    oq    -f     a^^  cos  t    +     b;j^  sin  t    4*     ao  cos  2t     +     b2  sin  2t 

■f     •   .    .     f     aj^  cos  nt    -f     b^  sin  nt  (17) 


n 


c(t)  ■=    \        aj^  cos  Irb    4"     bj.  sin  kt 


(10) 


1:  =  0 


It  is  desii'cd  to  choose  a,,  and  bj    such  that  the  ncan-square  error,   e(t), 

over  tile  irxtci-val  for  v/hich  f  (t)   is  defined  is  a  inininuEU 
^c  •»-  zTf 


e(t) 


-^  \i 


f(t)     -     c(t) 


'] 


dt   —    niniruLi 


(19) 


For  a  rJLniim.un,  the  partial  derivatives  of  e(t)  vd.th  respect  to  a,,  and 
b.  should  be  sero.  Tlie  value  of  g(t)  given  by  equation  (lO)  is  sul>- 
stituted  into  equation  (19)  •  Tlic  partial  derivative  vdth  respect  to 
a.  nay  be  talzcn  under  the  intecral  si^n  and  tlie  rccultinc  e:<pression 
equated  to  zero, 

*'^    "  -|p   n 

f(t)  -  \  (a^v  cos  L-t  4-  bj,  sin  Irt)   -  \   cos  jt 


1:=0 


J  L  j=o 


dt  =0  (20) 
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a     cos  !-b     cos  jt     f     b     sn  ::t     cc 


C   Jt        dt     = 

^         r(t)  Y^  COG  jt    dt       (21) 


"c  f   277"    . 2 


'c  j-0 

To  cvnluatc  tlie  lefl;  side  of  e-:uation  (21),   certain  oi-thoconal 

properties  or  the  gJjic  czid  cccine  sliovJLd  be  recalled, 
*c  f  277" 


X 
X 


cos  rcc     cos  n:    d::    «    0  for  n  ^  n  (22) 

c  f  277" 

CCS  nx    cos  nx    d:c    =  77"  for  n  =  n  (23) 


c  f  277^ 

cin  nil    cos  r.cc    d::    :=    0  for  all  n  and  n  (2I4) 


Tlie  onJiy  contribution  to  the  left  side  of  equation  (21)  occurs  vrfien  k 

and  j  are  equal.     Biuation  (2!^  nay  now  be  i.Titten  as 
n  n      re  +  277" 

1:«0  1:=0  ^  c 

As  a  renvlt  of  the  rearranncncnt  of  equation  (20)  to  f  ct  equation  (21), 
there  is  an  equalit7  between  tenns  vniich  ai^sc  frou  substitutinc  the 
saDe  value  of  k  on  each  side  of  equation  (2^).  Hence  the  general 
expression  for  a.,  is 

re  4-  277" 

a.^  =  -—/     f(t)  cos  Irb  dt       v.'hcre  k  =  0,  1,  .  .  .  n    (26) 

Tliis  is  reconnized  as  the  cencral  czq^ression  for  the  coefficients  of 
the  cosine  terns  in  tlic  Fourier  scries  ezq^ansion.     A  siinilar  process 
of  taldjp4j  the  partial  derivative  vrL th  res^^ect  to  h    iTill  yield  the 
general  expression  for  the  coefficients  of  tlie  sine  terns  in  the  Fourier 
scries  e:qxmsiono 


Hi 


The  rcnaiidjTC  quco"tion  to  be  Gcttlcd  is  ivhcthcr  tlic  valuoc  of  a., 
and  Ix,  juGt  obtained  represent  a  ija>niua,   :diiii,a,   or  ooxldlc  point.     Tliis 
is  accoi.iplislicd  Irj  findinc  tlie  value:;  of  the  tlirce  cecond  partial  dcri-^ 
atives  represented  by  A,  B,   and  C  at  the  poii.t   (a..,  b.  ).     Tl.c  dirfcr^ 
cntiations  naj'  ar;ain  be  carried  out  tinder  the  iiitccral  sign. 


a"e(t) 


^^  ^  ^^^n  n 


(27) 


D  =  f-^-i   (a,.,  b,,)  =  i/  0     dt   =  0  (28) 


(29) 


The  conditions  for  a  ixDlative  ninima  of  a  function  of  ti;o  variables  are 
B^    -     AC    <  0  (30) 

A    f     C    >    0  (31) 

Doth  conditions  are  net  in  tliis  case. 

Tlius  if  f(t)  be  picccr.vise  continuous  for  c  ^  t  ^   c     4-     27r>   then 
tlie  coefficients  of  the  partial  svun  of  ti:e  Fourier  series  of  f(t)  are 
precisely-'  th.osc  a^nonc  all  coefficients  of  tl:c  fuiiction,   c('t')j  trlrich 
render  th*e  aeai>- square  error  a  i-iiniinm.i. 

Tlie  foregoinG  proof  vra.s  su^ccsted  aj  a  nroblen  in  Ilildebrand-^^. 
Both  Guillcr.in"  and  I^plan       pix3ve  tlie  sane  result  usir^  different 
approaches. 
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CHAPTHi  II     All  EXTEISIOi;  OF  LUiVST  SviUARiiS  IT^IDICTIOH  TIIBORT 

1.     Bacic  Acsvj options. 

The  discussion  of  this  cl^aptcr  vrill  be  basca  uT:on  t;.c  ccncral 
problcn  as  stated  in  Ciiaptcr  I.     Tlie  received,   observed,  or  measured 
cicnal,   R(t),  iTl.ich  is  tlie  input  to  the  filter,   consists  of  a  true 
sicnal,   P(t),    and  noise,   r(t).     The  true  signal,    P(t),  v.Hl  be  assiuied 
to  consist  of  n  corr:lctc!l^'  forclniCTi-ni  iXmctions  of  tLue,   f-,(t),    fpCt), 
•   •    .   r  (t),    each  rjultiplied  by  a  constant  coefficient,   a-,,   a^,    •   .    • 
a  ,  v/hich  is  not  a  function  of  tine, 

P(t)    =    ^^i(^)     +     a.f2(t)     +     .    .    .     +     :2^f^('t)  (32) 

n 

P(t)    =    Y^   ah%(^)  (33) 

h  =  l 

ant 
Such  functions  as  e         and  siii  a^t  are  not  considered  sii'ce  thc^''  are  in 

effect  undcten.iined  functions* 

From  a  foreloiowledce  of  the  fom  of  P(t)  and  ovjr  operations  on 

R(t),  Y;e  attempt  to  obtain  a  closer  cstinate  of  P(t),    called  Q(t),    such 

tl:at  Q(t)  vrill  differ  from  R(t)  by  the  least  a:nount  in  so:.-.e  nanner  or 

other,     Tlie  "best"  estinatc^  Q(t),  T.lll  be  chosen  to  liave  the  sane  fona 

as  P(t),  tliat  is 
n 

Q(t)    «    V^   ^^vS-^^  .    (3U) 

Tlie  functions^  Ck('^)>  ^^  ^^^  e:q5rcssion  for  i^(t)  ai'e  closely  related  to 
the  functions,  fi^(t),  in  the  eiq^ression  for  P(t)  and  in  naj^r  cases  nay- 
be  identical.     The  ec^ct  restrictions  iiiposed  upon  the  functions  niaJring 
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up  P(t)   nr.d  Q(t)  -iTllI  be  diccucccd  ''r.  the  ncrrt  ccction.     Ik)  assi'orr^tion 
vrill  be  r^de  about  the  tvpo  of  noise,   !l(t).     It  vrQl  be  acsiir.cd  tliat  a 
roaGonable  criterion  of  '.icrronaancc  in  one  in  vr'.ich  t'r.c  tirjc  reinlitcd 
sqiuircd  error  is  roninicred  as  illustrated  in  section  tlirce  of  this 
chapter.     Tliis  r;ani:.iication  is  brov^ht  a'x)v.t  "by  tl^.c  vra^;-  in  v/hich  the 
undetcr:rined  constants,  b-j_,  boj    .    .    .  b  ,   arc  selected. 

In  order  to   si.:r)li:^'  tlie  plic/"sical  realisation  of  the  filters,   they 
will  be  assuncd  to  be  constructed  of  linear  elarientj  v/liich  do  not  vaiy 
v/ith  tii.ie.     11  .e  lirdted  scope  of  th.is  paper  vn.il  1  not  pennit  a  tliorouc^"^ 
discussion  of  a  broader  class  of  filters  vrirLch  nay  be  constructed  by 
permitting  tiic  sise  of  th.e  filter  elenents  to  vaiy  vrLth  tL-e. 

3h  ceneral,  vrc  assune  the  ftaiction,   P(t),  to  be  corpletely  foro- 
knorm.     There  are  c::ceptions,  hoivcvcr,  wliich  are  ai.ienp.ble  to  treatncnt 
even  if  there  is  sone  decree  of  uncei'taii-ty  as  tc  the  for-n  of  P(t)» 
Tliis  occurs  idien  the  function  is  unlrnovm  in  the  sense  that  it  is  one 
of  several  Ijiovm  possible  functions.     Tnc  filter  nay  be  designed  to  tal:e 
care  of  all  the  laiov-.n  possibilities.     As  an  c:ca:-.^lc,    si-.ppose  tliat  P(t) 
is  i:nawn  to  be  either  a-^,   a2  e  ,   or  a-^  e     •     Tl.en  the  actual  signal  for 
"VTliich  tlie  filter  is  designed  nicht  be 

Q(t)    =    bi    +    bo  e^    f    b3  e^t  (35) 

Tlic  price  paid  for  t!iis  uncertaintr/'  as  to  the  e:r.ct  function  present  is 
em  increase  in  the  noise  output  of  th.c  filter.     Tl'-is  is  so  because  the 
inclusion  of  additional  fimctions  vrLll  in  ccncral  require  the  filter  to 
have  a  greater  fl*equency  response  tlian  tliat  required  for  tlie  one  true 
signal  alone.     Tne  increase  in  filter  noise  output  liriits  the  c:rtcnt  to 
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AViU-c  thic  r.cti.od  can  be  cr.rriod.     Az-.othcr  case  ic  tiiat  of  the  uifxtciv 
:;Jjnod  fur.ctior.  a2_  cii'.  (-..-t  f  ao).     Ac  it  ctaiiclc,   it  cannot  be  trccted. 
TjOTTcvcr,   b;-  -  wttinc  it  in  the  equi\tilent  for,.,   l>j_  sin  \rt  f  bo  ccc  v.-t, 
it  i.r:^'-  be  conciclerod.     In  t!:ic  case  there  is  no  locc  of  ccnerality  nor 
increase  in  the  noise  output, 

2.     Applicable  Functions. 

The  oni^''  assomption  vriiich  -iTill  be  nade  alx)ut  the  tTpe  of  functions 
conprisinc  Q(t)   is  tliat  tlicy  fori  a  closed  set  vjider  a  linear  trano- 
lation  in  tine,  tluit  is,   no  new  functions  v/ill  be  ccnerated.     Tiiis 
restriction  results  f^n  the  fact  tliat,   for  z±rx^l±c±ty ,  the  filters  are 
considered  to  be  invariant  v.-ith  tL-ie.     In  co^"C  fr*on  a  static  case  of 
least  squares  cui've  fitting  to  the  c^Tianic  situation,   no  tine  reference 
is  provided  the  filter.     Given  a  sinusoidal  type  y.nvc  tliat  lias  been 
Coin{:  on  for  an  indefinitely  long  tii.ie,   one  cannot  say  v/hether  tlie  vrave 
is  a  sine  or  a  cosine  luiless  sone  reference  tir.ie  is  chosen.     Ilowever, 
no  natter  ;That  reference  is  chosen,   the  vr?.vc  can  ali/ays  be  described  as 
sone  linear  conbination  of  cos  vrb  and  sin  \rt.     To  understand  vrliat  tliis 
restriction  inplies,   consider  the  fiaictions  b^  sin  v/t  and  b2  t    under  a 
translation  in  tii.'^  of  duration  c. 

b^  sin  w(t  f  c)    ~    b^  sin  vrt    -f     b^  cos  irb  (36) 

bg  (t  4-  c)2    =    b^^     f     b(;  t    +     b^  t2  (37) 

In  both  cases,  new  functions  resulted  from  the  translations  in  tine. 
IIo\T  if  the  c:::.rcssions  on  tl:e  richt  sides  of  equations   (36)  and  (37) 
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imdcrco  a  further  translation  iii  tirnc,   tJic  translationa  Tfill  acrely 

cliance  the  coefficients  vdtho-ut  ccncrr.tinc  aiX'"  nc?v;  functions.     Tiius 

the  ri^lit  sides  satis^  the  recfuirencnt  for  functions  vrhich  nal:e  up 

Q(t).     It  riay  also  be  observed  that  functions  of  the  form  b,   t  *     and 
1 


can  never  satisiy  tlds  restriction. 


b-j_  •♦•  b2  t  f  b^  t' 


The  ne^  question  reqirLrinc  an  ans-iTcr  is,   "\'fhat  functions  remain 
a  closed  set  under  a  linear  transfomation  in  ti:.ie?"     As  an  illustration* 
consider  the  case  in  vrixLch  th.ere  are  t^vo  functions  involved. 


Q(t)    =    b^^  ci(t)     f     bg  C2(t) 


(38) 


Tiie  linear  transforL:ation  mny  be  Tnn.ttcn  in  iiatrix  form* 


gl(t  f  At) 

A 

B 

Ci(t) 

C2(t  f  At) 

C 

D 

22^^) 

But  A,  B,    C,   and  D  must  be  ciiosen  so  that  vhen  ^t  equals  zero 


Cl(t  4.  0) 

1 

0 

Ci(t) 

G2(^  +  C) 

0 

1 

C2(t) 

Tliis  condition  is  satisfied  if 


G-L(t  f  At) 
g2(t  +  At) 


1  4-  a  At         b  At 

c  At  1  f  d  At 


Ci(t) 
So(t) 


(39) 


(UO) 


(la) 


TTiiere  a,  b,    c,   and  d  are  arbitrary  constants.     Subtract  tlie  natrix 

f3X)iii  both  sides  of  the  natriic  equation  {hX)  and  divide  "by  At. 


Ci(t) 


g^^t) 
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c^U  4- At)  -  c^jt) 


At 


a 


Ci(t) 


Co(t) 


(U2) 


Since  this  nust  hold  for  all  At,    lot  At  approach  tlic  limit  soro« 
Tlie  rcsultine  derivative  is  dcnotod  by  a  prime. 


Ci'(t) 


a        b 
c         d 


Ci(t) 


(13) 


!ibT7  talrinc  the  second  derivatives  rdth  recpect  to  tiine. 


1 1 


1 1 


(t) 
(t) 


a        b 
c         d 


Ci'(t) 


(U;) 


Tlic  matrix  equations  (U3)  and  (Uli)  nay  novr  be  used  to  find  similar 
second  order  differential  equations  in  g-.{t)   and  g^^^^*  ^icse  equations 
tal:e  the  fom 

6"(t)  +  C-L  g'(t)  •♦•  C2  c(t)  =  0  vrherc  C^  and  Cg  are  constants  (15) 

By   ucinc  si.ni.il  ar  methods,  it  is  evident  th.at  if  Q(t)  consists  of  n 
functions,  these  functions  must  be  solutions  of  the  reduced  equation  of 
an  n  th  order  linear  differential  equation  Trith  constant  coefficients. 
Tliis  restriction  is  not  as  serious  as  it  mi(;;ht  firct  apT">e^^  since  a  iTide 
variety  of  useful  functions  belong  in  this  cro"*^!"*  Included  are  esqao- 
ncntials,  polynomials,  sinusoids,  and  functions  formed  by  adding  or 
multiplying  these  types  together.  THiile  the  fimctions,  f(t),  that  fom 
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the  tnie  cicml,    P(t),   Liay  be  concidcrcd  to  be  aix/"  one  solution  of  the 
differential  equation  such  as  a-^  sin  vrt,   Q(t)  nust  consist  of  t^^e  corro- 
sponding  coiqilete  set  of  functions,  b^    sin  T."t  +  b2  cos  v.-L.     Sinilrxly, 
if  f(t)  lias  a  ix)lynoni:.l  tern  such  as  a2  t^,  c{t)  must  contain  the  corro- 
spending  co:.plete  polynor.iial,   b^fbj^t  +  b^t. 

3«     Tine  Weichted  Least  Squares  Curve  Fi-ttinc. 

Tne  process  v;}:ich  i-.lll  be  used  to  obtain  a  tine  vrtsichted  least 
squares  fit  is  illustrated  in  Ficvorc  h,     Tl^c  top  sketch  shows  tlie  noisy 
signal,  R(t),   and  tlie  best  estimate  of  tlie  true  signal,   Q(t).     The 
difference  bctu'ecn  the  tvro  signals  is  the  residual  error,   R(t)  -  Q(t). 
In  the  niddle  of  the  figure  is  th.e  square  of  this  residual  error, 
lR(t)  -  Q(t)   '^»     In  order  to  allo-.v  sone  latitude  in  th.e  constmction  of 
the  filter,   tliis  squared  error  is  ;7cightcd  over  the  past  in  sone  arbi- 
trary nanner  by  nultipl;,'inc  the  square  of  the  residual  error  Ir^^  a 
Tfeigiiting  function  of  tine,    Ll(t),     Tl^e  resiJ.t  of  the  multiplication 
is  to  produce  a  tine  vreighted  squared  residual  error  curve  as  sh.ovvn 
\jy  the  crossliatched  area  of  the  botton  diagran.     Tlie  ai^a  underneath 
tliis  curve  is  given  by 

A    =    /         rR(t)  -  Q(t)l"  LT(t)  dt  (16) 

vZ-oo 

It  is  preciccl;.'  t!  ds  crossliatch.ecl  area  vldch  vriU  be  nininised. 

Tlie  squared  enxDr  vreightir^*  fimction,   !.:(t),  Dsy  be  tliought  of  as 
tlic  niter's  ncnoiy  of  the  squared  error,     Tlie  r.vithenatical  restriction 
on  ll(t)   is  tliat  it  be  chosen  so  tliat  the  area  integral  is  fii-ite.     There 
is,  liov/ever,   another  practical  restriction  on  :.l(t).     Since  one  rmct  v;ait 
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Ficure  U.     TLjC  V/eiglited  Least  Squai'es  Curve  Fitting. 
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uiTtil  the  £3rx)otliinc  tinie,  T,  lias  becjn  con^jletod  or  imtil  the  filter  has 
had  tiue  to  settle  before  it  provides  an  "oiTtinuun"  output,  the  na^ooiy 
of  the  filter  cannot  ezrbend  too  far  into  the  past,     l^t)  nay  be  a  conv- 
promise  between  a  long  snootliinc  time  v/hich  laalccs  use  of  more  information 
about  the  sicnal  and  a  short  tine  vihich  Trould  provide  a  quicker  decision 
as  to  the  nature  of  the  signal. 

Probably  tlie  tliree  nost  ir^^ortant  forms  of  M(t)  are  the  rectangular, 
e:q5onential,   and  danq^cd  sinusoidal  types  shovm  in  Figui*e  5»     In  tlie 
rectangular  type,  the  filter  \reights  all  squared  errors  from  a  ccjrtain 
fi:ccd  tine  in  the  past,   T,  up  to  the  present  tine  oquaUy.     Tills  corro- 
spends  to  the  v/eighting  discussed  by  Zadeli  and  Ragaazini-^     in  which 

M(t)    =    u(-t)  -  u(-t^T)         Tdiere  u(t)  is  the  unit  step  function         (U?) 

Tlie  abrupt  clian^e  in  Treighting  results  in  filters  "v^iich  contain  a  delay 
line  try-pe  elcnent.     Another  useful  type  filter  is  one  which  forgets  -what 
has  gone  on  in  the  past  e^?:)onentially.     Ejiponcntial  vreighting  is  defined 

U(t)    =    e"^/^  u(-t)  (U8) 

Tills  type  of  nenory  is  very  prondsing  in  that  it  corresponds  to  tlie 
natural  da;.?)ing  foiuid  in  nost  pi^'sical  cystens  vrith  dlsslpative  elements* 
and  hence  holds  forth  the  hone  that  the  filters  nay  be  realized  vrLth 
luiriped  constant  elements.     Tl;e  third  type  oi  v^eighting  defined  ty 

M(t)    =    e^/^  (cos  \rt  -  sin  vrt)  u(-t)  (1-9) 

is  an  attenpt  to  appro:d.natc  tiie  rectangular  filter.     Tlie  output  of  tliis 
filter  ivould  tend  to  liave  a  transient  dacjped  oscillation  and  also  holds 
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0 


-t 


:^:.-\i-c  :>•     ^'viccJ.  Stiur.red  lirci'  :c..<.i-^  IVjictl-u-.G* 


rJ: 


forth  the  hope  of  realisation  T/ith  liuiK)ed  constant  elements.     Otlier 
types  of  vreichting  Trtiich  nieht  find  application  under  certain  circunv- 
stances  include  t  e  '    ,  \r  e  '    »  and  e  '^  sin  vrb. 

U*     Fornnilation  of  the  Wcichtinc  Function. 

If  the  c:q;^resc;ion  for  Q(t)  given  in  equation  (3U)  is  substituted 
into  equation  (I4.6),  tlio  ecxprcssion  for  the  crossliatched  area  in  FLcure 


n  -. 


n 
R(t)     -      )        b,,  g.  (t)|      M(t)  dt  (50) 


Tlie  conditions  for  a  mir-iiiun  are  that  the  partial  derivatives  of  the 
expression  for  the  area  vrith  respect  to  the  coefficients  b. ,  bp*  •  .  . 
b  equal  zero.  Tliis  results  in  a  set  of  n  sinultaneous  linear  equations* 

^^  =  0      Tvhere  k  =■  1,  2,  ...  n  (5l) 


»\ 


In  order  to  carry  out  these  differentiations  under  the  integral  sicn,  a 
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form  of  Leibnitz's  rule  sinilar  to  that  given  by  Kaplan   '  is  noiv  stated. 

Let  f  (x,  t)  be  continuous  and  have  a  continuous  derivative  df/dx  in 
a  domain  of  the  xt  plane  Tfrioich  includes  the  rectangle  a  ^  x  S_  b, 
t,  ^  t  ^  tp*     Tlien  for  a  <^  x  <  b 

i.  /      f(x,  t)  dt    =    /       i^  (x,  t)  dt  (52) 

dx/  /        dx 

The  partial  derivative  vrith  respect  to  the  general  coefficient,  b.  ,  vrLll 
be  talren  under  the  integral  sign. 


^ 


R(t) 


'-00 


n 


M(t)  dt 


1^ . .  /  «|  -" 


n  p  n  -|^ 

j-1  ■-  1:-1  -^J 


dt 


(^3) 


(M 


Tlie  clianee  of  srabocript  fron  k  to  j  T/as  ncic^e  to  indicate  that  the 
sunmations  are  distinct.     Sctti-X'  the  partial  derivative  eqiial  to  zero 
and  rearrancinc  the  resulting  equation 


'-co  j.l 

IT  trro  nc\7  EP^oabols  are  defined  as  follcfVfs, 


h    = 


R(t)  g^(t)   M(t)  dt 


'-O0 


Sjk   =    %:a  — 


Ck(^)  gj(t)  *)  dt 


(56) 
(57) 


-co 


then  BjjCan  be  evaluated  since  g^(t),  C-j("t)>  c-'tcI  M(t)  are  all  loiovin, 
and  the  set  of  n  simultaneous  equations  defined  ty  eqr^ti:'n  (55)  tike 


tlie  sir^ler  fom 
n  JQ 


E^o  =  E  C^j'i^^ 


(58) 


d-1 


b-i    j»-j 


In  natrix  fom  these  equations  become 


h 


h 


In 


Bn 

%2 

.  .  .  a^n 

^ 

^21 

Boo 

•    •    •  ^2n 

Bnl 

Sn2 

•   •    •  ^ 

bn 

(59) 
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Let  Cj^  ^c  the  inverse  intrir.  of  Bjij,,  th^t  is 


^11  ^12 
C21  C22 

•  •  •  ^?_n 

•  •  •  >-''^ 

1 L       t-L. 

.  .  .  B]j^ 
•  •  •  ^2n 

Cnl  Cn2 

•  •  •  ^\m 

Dm  Bn2 

•  •  •  %n 

1       0       . 
0       1       . 


Prcnultiplyinc  botl:  sides  of  equation  (59)  4/"  "t-^-c  Cj^  matrix 


Cr^      Cq_2   •  •  •  C^j^ 

^1 

lo 

•   • 

^ 

•      • 

or,   in  tlie  form  of  a  s\unr.iation# 


n 


^:=^      Y^   ^:i  ^i 


0 

0 


0       0       •    .    .     1 


(60) 


(61) 


(62) 


i«l 

Siibotitutinc   xroo  equation  (62)  ir.to  equation   (3U) 
n  n 

Q(t)   ^    Yl     Y^   ^^  ^^i  ^k^^^  ^^^^^ 

k-1     i=l 

To  avoid  confusion  vrith  the  actual  variable,  t,  tlie  dujxj;-  variable  of 

intecration  in  equation  (^6)  is  changed  from  t  to  x»  The  rcsultinc 

value  of  li  is  substituted  into *  equation  (63)* 
n    n 


Q(t) 


-EE 


C]d  gk(t') 


(6U) 


R(x)  gi(::)  li(x)  dx 
k«l  i«l  ^-°o 
Rearraneing  equation  (6h)  so  tliat  all  terns  are  vji^^cr  the  integral  sign^ 

/O   n    n 
YZ  ZZ  ^^"'^  ^^^"•^  ^^-^  ^^^^^  '■"^"'*  *"         ^^^^ 

-00  k«l  i«l 
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Let  C]H  ^-c  the  inverDe  i:ntri::  of  B^ti-*  that  is 


j»k 


Gil     ^12 
C21     Coo 

•  •    •   Ctj, 

•  •    •   C^ji 

^11     ^12 

1— !.           tt 

•   •    •  Bin 
.    .    .  B2n 

Cnl     Cj^ 

•    •    •   ^ 

Dnl    Bn2 

•    •    •  ^ 

1       0 
0       1 


0 
0 


0       0       .    .    .     1 


(60) 


Prcnultiplyinc  botl:   sides  oT  equation  (59)  ty  the  Cj-j_  matrbc 


^     "^12     •   •   •   '-'In 
^21     ^:.2     '    •    •   ^2n 

^nl    ^n2     •   •   •   ^nn 

h 

•      • 

= 

"1 

bo 

•       • 

(61) 


or,   in  tJie  fom  of  a  svonr.iation^ 
n 

i«l 


(62) 


Substitutinc  fron  equation  (62)  ir.to  eqiiation  (3U) 
n  n 

Q(t)   "    Yl     Y^   ^^  ^'i  ^k^^^  ^^^^^ 

k-1     issl 

To  avoid  confusion  v/ith  the  actual  variable,  t,  th.e  duixy  variable  of 

intecration  in  equation  {%)   is  chanp:ed  from  t  to  x.  The  recultinc 

value  of  I^  is  substituted  L'lto *  equation  (63)  • 
n    n   |—  — 

Q(t)  =  Y^  Y2  I  ^^^^  ^^^''^  "^"^  ^  ^^  ^^^^^        ^^^^ 

1:«1     i«l    L^~«> 
Reai*rancing  equation  (61;)   so  tliat  all  terns  are  vjidcr  the  integral  si{jn^ 

/•  O      n  n 

Y"     Y^  R(::)  Ci(::)  0,,^  Ck(t)  I.l(x)  dx  (65) 

-00     i:«i     i  =  i 
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Consider  that  Q(t)  is  the  output  of  a  niter  vrith  an  input,  R(t)« 
Rcferrinc  to  Ficure  h,  it  ic  seen  tliat  Q(0)  is  the  best  estimate  of 
the  true  sicnal  at  tlio  present  tir.:e»  If  t  is  rKJsitive,  the  value  of 
Q(t)  is  an  ectinate  of  a  future  value  of  the  true  cignal.  Thus  the 
quantity  t  is  the  prediction  tiiie.  For  the  fixed  filters  under  con- 
sideration, let  t  bo  a  fi:ced  prediction  tine   designated  ^j   the  sj/nlx)! 
©•  Etiuation  (6^)  037  novf  be  interpreted  as  a  convolution  intecral  of 
the  form  shoi-m  by  equation  (66)» 


Q(e) 


W(e,  0  -  x)  R(x)  dx 


(66) 


'-00 


A  coi:parison  of  equations  (6^)  and  (66)  indicates  that  the  weichtinc 


function  of  the  filter  must  be 

n  n 

W(©,   0  -  x)    -  2_2    Y~\     ^i^^^  ^  ^^^®^  "^''^ 
k=l     i=l 

Ifeidng  the  substitution 


(67) 


0  -  X 


(68) 


the  "VTeichtinc  function  of  the  filter  becomes 
n  n 

w(e,  t)  ^  Y2,  Y^    ^i^"^^  ^^  ^k^®^  ^"^^  ^(*) 

k«l    i=l 
Trticre  u(t)  is  the  unit  step  function.     In  matrix  form  this  becomes 


(69). 


w(e,  t) 


gl(-t),  g2(-t),   •  .   .  Ej,i"t)\ 


- 

■•— 

^"               "■ 

<ai 

*a2 

•      •      •      ^ 

gi(e) 

^21 

C22 

.  .   .  C2j^ 

C2(®) 

Cnl 

Cn2 

•   •   •  ^ 

•     •     • 

M(-t)  u(t) 
(70) 
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Since  most   nctvrorl:  synthesis  is  done  in  the  frequency  domain,  it 
is  convenient  to  represent  the  filter  try  its  conplex  frequency  transfer 
function,  H(s),  the  Laplace  transform  of  the  weichting  function. 


ao 


H(s)  =  /    W(B,  t)  e""^  dt 


(71) 


To  obtain  snooth  derivatives  or  intecrals,  H(s)  is  either  nultiplied  or 
divided  by  tlie  variable  s* 


5»  Steai^  State  Filter  Pa3six)nsc  in  the  Absence  of  lloise. 

If  the  filter  iveishtiiig  functions  found  in  the  preceedinc  section 
are  valid,  a  proper  input  to  the  filter  in  tlie  absence  of  noise  should 
produce  a  steady  state  output  ivitli  no  error.  Since  the  noise,  N(t),  is 
zero,  the  input,  R(t),  is  equal  to  the  true  sicnal,  P(t)#  From  equations 
(1)  and  (33)j  the  input  may  be  eiqDressed  as 


R(t) 


n 

h=l 


^h  ^h(t) 


(72) 


Tlie  filter  vreighting  function  is  civen  ty  equation  (69)  as 

n    n 

w(e,  t)  =  V^  V^  Ci(-t)  Cj,i  gi,(e)  u(-t)  u(t) 


(73) 


k-1  i«l 
From  equation  (5)j  the  steady  state  output  of  tlie  filter  is 

Q(t)  —  /   v;CiO  R(t  -r)  dt   =  /   w(-tO  R(t  f  r)  dt 
^o  J  oo 

Substituting  the  expressions  for  the  input  and  Treighting  function  into 


(7U) 


the  equation  for  the  Gtea(^''  state  output,  one  obtains 


Q(t) 


n    n 


gi(t)  c^d.  ek(»)  "(^) 


-oo 


1:»1  i-1 


n 


2~J  ®h  fh(^  *♦•  ^ 


haal 


dt       (75) 
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n 


Assunc  tliat     >        fj^Ct  f  XT)  may  be  e^qjressod  as 


h.l  n 


h-l  h'«h    p*l 

If  equation  (76)  is  subartituted  into  equation  (7^)»  tlicn 
n   n   n   n 

k«l  h«l  i«^l  p«l  ^<»  ^"^^ 

From,  equation  (57)  the  intccral  B^^  nay   be  defined  as 


^ip  -  f        gi(^  6p(^  "(^  ^^ 


B,„  -  /    g^  (D  g^(tO  M(r)  dr  (78) 

00 


This  "vrill  reduce  equation  (77)  to  the  fom 


n        n  n        n 


Q(t)  "    }_,  l_^  l_^  l_^      ^  ^ki  Sk(®)  %  I>1?)(^)  (79) 

1:«1  h=l     i-1  p  =  l 
Tlie  relation  bctvrcen  B^p  and  0^^  is 


n  n  n 


(80) 


/       Bip  Old.    -    1  ^"^^  p  =  k 

=*   0  Y/hcsn  p  ^  k 

Bjuation  (79)  nay  nmv  be  reduced  to 
n  n 

Q(t)   ^   YZ    YIj    ^^^^^  ^^  ^^^^  ^^^^ 

k«l     h-l 

"By  conparison  vrith  equation  (76) 


(82) 


k-l  h-l  h=l 

so  that  Q(t)  can  be  expressed  as 
n 

Q(t)  -  y2     ah  fh(t  4.  9)  (83) 

h«l 
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!n^-i3  e^q^recsion  nay  novr  be  coiToarod  vrith  cQu:ition  (72)  to  c^vc 

Q(t)    —    K(t  ^  e)  (OU) 

Tims  tlie  steady  ctatc  oiitimt  of  the  filter  equals  the  filter  input 
displaced  ty  the  amount  of  the  prodiction  tine,  ©• 

The  orOy  assumption  nadc  in  tliis  section  v/as  that  defined  by 
equation  (76),  It  leads  to  a  set  of  equations  vrry  sirailar  to  the 
natri:c  set  civcn  in  equation  (39 )•  Tlds  assuinption  corresponds  to 
the  roquirenent  that  the  out  pi' t  of  the  filter  have  the  same  fom  as 
the  input  except  for  a  possible  translation  in  tLr.e.  One  nay  a^^ajji 
conclude  tiiat  the  functions  vniich  axe  anenable  to  treatment  arc  the 
solutions  of  the  reduced  equation  of  an  n  th  order  linear  differential 
equation  vdth  constant  coefficients. 
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CII&PTIE  III     REALIZATION  OF  SLDOTHIIIG  AlID  PREDICTIIIG  FILTIIIS 

!•     liBthods  Available  for  Construction. 

The  fon.tulation  of  the  iveichtinc  iXmction  of  Cliapter  H  did  not 
insure  tliat  every  filter  vras  pl^sically  realisable  ucinc  lun^ped  conctant 
elcncntc.     It  did,  hOTrevcr,  provide  cuch  a  vrlde  variety  of  squared  error 
inenory  functions,   U(t),  tliat  a  high  probability  e:d.sts  tliat  the  filters 
nay  bo  constructed  by  one  of  several  methods* 

V/hcn  a  roct angular  error  ine-Tory  function  is  used,   it  is  often 
convenient  to  fr^mthesize  tlie  transfer  functions  xrith  operational  ampli- 
fiers such  as  those  used  in  analog  corB^utcrs.     Tliis  is  especially  true 
if  tlie  signal  under  consideration  is  con^josed  of  lo;v  frequency  coi>- 
jMDnents  only.     Since  the  rectangular  error  r.ianory  function  leads  to  a 
delay  line  element,  the  operational  amplifiers  liave  the  advantage  of 
providing  isolation  and  a  means  of  sunr.iing  the  undelayed  and  delayed 
signals.     "Wliile  conventional  delay  lines  could  be  used  for  shorter 
smootiiing  times,   it  v/ould  probably  be  necessar,^  to  resort  to  a  contin- 
uous tape  or  drum  arrangement  such  as  the  one  sliovm  in  Figure  6  for  the 
longer  smootiiing  times.     Tlie  actual  synthesis  of  tlicsc  filters  iTill  not 

be  discussed  in  detail  since  the  needed  information  may  be  found  in 

20 
books  on  analog  computers  such  as  t^.c  one  cy  Kom  and  Kom     . 

If  an  exix)nentictl  or  damped  sinusoidal  error  memory  function  is 

used,   it  is  often  possible  to  syntliesise  the  f^.ltor  as  a  tiTO  teninjial 

driving  poiiit  irpedance,   Z(s).     The  in^iit  to  the  filter  is  a  current 

•wiiich  could  be  supplied  from  a  pentode  T/iiilc  the  output  is  the  voltage 

across  tlie  terminals.     A  driving  point  impedance  must  be  a  positive  real 
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Erase 


Fic"UJ^G  6.     Ifagnetic  Tape  or  Dnun  Delay  ELcment. 

function,     Tlie  v.'ell  lnio\-,Ti  conditionc  for  a  rational  IVuiction  to  be  a 

positive  real  function  are 

^(s)  is  real  ivhcn  s  is  real. 

Z(s)  is  anal;>'tic  in  the  richt  half  plane 

Poles  on  the  boiindar,'"  are  sir^ole  vrith  positive,'   real  residues. 

r^p(j^0j    ^  0  v.t.cn    Re  Is  I  ^  0 

If  the  transfer  function  meets  these  conditions,  the  syntl^'.esis  of  the 

filters  as  driving  point  ii.Tpcdances  nay  often  be  done  by   inspection, 

and  can  alv.'ays  be  acco:.Tr:>lishod  by  the  methods  of  either  Binme^  or  Bott 

and  Duffin^»^°. 

A  second  netliod  for  synthesizinc  the  transfer  function  is  available 
Trfien  the  e:x:fonential  or  damped  sinusoidal  error  naTory  function  is  used. 
The  transfer  function,   n(s),  may  be  considered  to  be  the  ratio  of  the 

open  circuit  output  voltage  pi  a  tvro-teminal  pair  netTrorl^  to  the  input 

1^ 
voltage.     Kalial  '    has  shcv/n  the  conditions  for  realizing  the  transfer 
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function  to  to. thin  a  constant  factor  using  a  finite  ntmbcr  of  limped 
constant,   linear  elements.     For  the  syrnnctrical  lattice  nctvrorl:  tlicse 
conditions  are 

H(s)  is  analytic  in  the  right  half  s  plane. 

Poles  on  the  boundary  arc  siiaplc  "vriLth  residues  having  a  zero  real  part. 
It  is  evident  that  a  vdLder  class  of  functions  may  be  treated  by  this 
netliod  than  in  the  case  of  driving  point  irroedances.  However,  there  is 
a  practical  problen  vrtiich  arises  from  the  fact  tliat  no  comnon  ground 
exists  between  the  input  and  output  terminals.  Tliis  difficulty  vrill  not 
be  present  if  the  syntlicsis  is  carried  out  as  a  ladder  netvrork.  Kahal-^^ 
has  shovm  tliat  one  additional  condition  besides  those  for  the  j^rmnetrical 
lattice  must  be  satisfiod  to  realize  tlie  transfer  IXmction  to  -within  a 
constant  factor  as  a  ladder  nctvrork.  TliLs  condition  is  tliat  H(s)  must 
have  no  zeros  in  the  right  lialf  plane,  A  najor  problem  in  the  actual 
E^mthesis  of  the  filters  is  usually  the  factorization  of  the  transfer 
function.  Tvro   methods  for  canning  out  the  actual  synthesis  of  the 
filter  are  those  of  Darlington^,  and  Weinberg^'''*  ^". 

2.  Filter  Design  Procedure. 

Before  proceeding  vrith  exan^ales  of  specific  filters,  it  would  be 
well  to  list  the  general  steps  to  be  follo^-.-ed  in  the  design  of  any 
filter.  Usually  the  ftmction(s)  to  be  smootiied,  the  prediction  time, 
and  the  functional  desired  are  deterrdned  by  the  requirements  of  the 
problem. 

Step  1.  Frou  the  known  function(s)  to  be  smoothed^  find  the  complete 
set  of  functions  "VYhich  form  a  closed  set  under  a  translation  in  tine. 
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If  the  STTDcthed  integral  of  a  polynonial  is  desired,  the  filter  nust  be 

desicncd  for  the  next  liighcr  order  po]lynonial.     Conversaly,  the  an^Dothod 

derivative  of  a  polynonia].  Yd.ll  be  designed  for  the  nei±  lower  order 

polynonial. 

Step  2.     Qioose  a  squared  error  nenory  function,    M(t)» 

Step  3*     Find  the  elements  of  tlie  B^  matrix  in  accordance  rrlth  equation 

(57)- 

Step  U.  Find  the  C[^  matrix  vrhich  is  th.e  inverse  of  tlie  B-jj^  matrix. 

Hetmds  for  obtaining  tlic  inverse  matrix  are  given  in  the  mathematics 

texts  by  Pipes   and  Guillcnin  • 

Step  S»     Find  the  nuncrical  values  of  the  functions  of  step  1  at  the 

prediction  tii:o,  G, 

■Step  6,  Determine  the  filter  Treighting  function  as  given  by  equation 

(70). 

Step  7«     Talre  the  Laplace  traiisfcrm  of  the  vreigl'itrjig  fiuiction.     liiltiply 

the  result  by  s  if  a  derivative  is  desired  and  divide  it  by  s  if  an 

integral  is  desired. 

Step  o.     Use  the  cor.plex  freqiienGr."  transfer  function  found  in  step  7  as 

tlie  basis  for  the  s^oithesis  of  the  net\TOrk. 

3.     Smoothing  and  Prediction  of  an  Eqoor.er.tial  Function. 

The  only  functions  Trhich  arc  solutions  of  the  first  oixier  linear 
differential  equation  rrlth  constant  coefiicients 

f «     f     Ci  f   »    0  vriiere  C^  is  a  constant  (85) 

are  of  the  type  a^  e     .     Tlie  steps  of  the  prececding  section  vrLll  be 
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follovrod  to  find  filters  wliich  both  crcoth  and  predict  for  tliio  function. 

For  the  first  case  it  is  assunod  tiiat  rectangtilar  vreichtiiig  of  the 
squared  error  is  to  be  used  to  snooth  the  noisy  function  a^  e  •  The 
best  estinate  of  the  function  itself  is  desired  v/ith  a  prediction  time 
of  9  seconds  and  a  snoot^dng  time  of  T  seconds. 


Stet)  1. 


Si  = 


nfc 


Step  2.    U(t)  —  u(-t)  -  u(-t^T) 
Step  3*    B-Q 


-r 


step  I',,    Cii  = 


2m 


1  -  G 


-2m 


—  ^m® 


Step  S,         Ci(©)  =  e 


Step  6.    W(t)  = 


Step  7.    Il(s)  = 


2m  e' 


n(9  -  t) 


1  -  e 


-2raT 


[^(t)  -  u(t  -  T)] 


2m  e' 


loe 


1  -  e-2^T 


1  .  e-^e-*^^ 


(s  4-  n) 


(86) 

(37) 
(88) 

(89) 

(90) 
(91) 

(92) 


Step  8,  One  possible  method  of  s;:,aitliesizinc  the  transfer  function  is 
shoT/n  in  Fi^jure  7»  Operational  aiiplificr  elements  arc  indicated  by 
triangles.  Actual  values  of  the  elements  are  not  shonn  since  they  depend 
upon  ©,  T,  the  type  of  operational  amplifier  used  and  the  anticipated 
signal  svrlng.  Either  the  delay  element  or  its  associated  operational 
amplifier  must  provide  a  negativ^e  output  v/lth  respect  to  the  input  in 
order  tliat  the  final  sumi-jing  aj^plifier  may  perforr.i  the  subtraction 
indicated  in  equation  (92) • 
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^Vv^A/^ 


TT^Sm^ 


E  in 


-^V\AAA 


rv\AAA^ 


Fieiirc  ?•     E;<poncntial  Predictor  -ivith  Rcctanciilar  Weichtinc 


The  second  case  to  be  illustrated  vdll  be  siiTllar  to  the  Urst  -with 
the  exception  tliat  exponentioJ.  v.'eichtinc  ttLII  be  used. 


Step  1.         g3_  «    e^ 

Step  2,         H(t)   «  eVT  u(-t) 


Step  3.        B]_]_  = 


.2rrt  et/T  dt  - 


2r.  t  l/T 


(93) 
(9U) 
(95) 


Step  U.         C]_i  a=   2ri  4-  1/T 


Step  5.         Ci(e)   =  e^ 


Step  6.        W(t)   =  e^[2n  4-  I/t]  e'^"^-^  VT)  t  ^(^,) 


.r:© 


step  7.        Z(s)  =  fZJ^±2fll 


(96) 
(97) 
(98) 
(99) 
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step  G,     Tlic  G7ntlic3iG  ol  tlic  flltor  as  a  drivinc  point  iqpodance  i: 
illustrated  in  Fic^ire  8« 


Plcure  8,     E-qjoncntial  Predictor  v/itli  Exponential  Weighting 

It  r.iay  be  noted  tliat  vriien  la  is  nq;^ative,   the  \'alucs  of  T  7/hich  -will  give 
a  realisable  netvrorl:  are  restricted.     Tiiis  iias  plrysical  significance  in 
that  it  is  necessary  for  the  noise  output  to  deco^'  at  a  faster  rate  tiian 
the  signal  for  a  usable  output. 

U«     Snoothing  of  Sinusoidal  Functions. 

Tlie  functions  a-j_  sin  Trt,  a2  cos  irb,  and  the  linear  combination  of 
the  trro  are  solutions  of  the  second  order  linear  differential  equation 
Tdth  constant  coefficients 


f»'     +     Cif     +     02^="^ 


Tfhere  0^  a.nd  C2  arc  constants 


(100) 


In  the  follovring  e:^r.ple  a  noisy  sinusoidal  function  vrith  an  angular 
frequencTy'  of  one  radian  per  second  is  to  be  snoothed.     llo  prediction  is 
desired  and  the  error  vreighting  is  of  the  e>q-onential  type  vrlth  a 
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SEXDOthirii];  tirac  constant  of  one  cecond. 


Step  1.    Ci  =  coc  t 


atep  3» 


Step  U« 


22 


=•  sin  t 


Step  2.    M(t)  =  e*  u(-^) 


3/^   -1/^ 

2    1 
1   3 


(101) 
(102) 
(103) 

(loU) 


step  5.    g3^(0)  =  1 

Step  6.    W(t)  =  (2  cos  t  -  sin  t)  e""^  u(t) 


Step  7.    Z(s) 


2s  +  1 


s""  f  2s  f  2 


(105) 

(106) 

(107) 

(108) 
(109) 


Step  8.  Ficurc  9   shows  the  synthesis  of  tliis  filter  as  a  driving  point 
icpedance. 

As  nicht  have  been  anticipated,  the  filter  is  a  resonant  circuit. 
At  an  angular  frequency  of  one  radian  per  second  its  iiipedance  is  purely 
resistive  and  equal  to  one  olin.  The  size  of  the  elenents  should  cause 
no  alarm  since  they  depend  upon  the  desired  frequenc:.'"  and  iripedance  level 
of  the  filter.  For  very  low  frequencies,  it  Might  be  convenient  to  use 
the  meclianical  analogues  of  the  electrical  elenents. 
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Ficure  9»     Filter  for  aaootiiinc  Sinusoidal  Functions 

It  appears  possible  to  constnict  a  filter  for  recurrent  square  or 
savrtooth  v/aves  by  designinc  tlicm  for  tlic  fundar.cntal  frequency  and  a 
suitable  number  of  tlic  proper  hxiraonics.     A  major  difficulty  with  this 
nethod  is  tliat  of  fiiidinc  the  L-. verse  of  a  natirb:  vriith  a  large  nun'oer  of 
elenients* 

5«     PolynoriLal  Filters. 

As  an  e:ca:.iple  of  the  design  of  poQ^^-noiiial  filters,  the  steps  for 
constructing  a  filter  vrhici:  smooths  arjd  predicts  a  first  order  poly- 
nomial or  ra_?)  vd-ll  be  siioc.Tn,     Tl.e  error  vreighting  is  of  the  eyqx)nential 
tgnoe  7ri.th  a  snoothing  time  constant  of  one  second. 


Step  1, 


Si-  J. 
52  ~  "^ 


(110) 
(111) 


UO 


I 


step  2.         n(t)   -   e"^  u(-t) 


(112) 


Stop  3» 


Step  !;• 


H- 


1  -1 
-1  2 

2  1 
1  1 


Step  5.         Ci(©)  =  1 


g2(e) 


Step  6.         ;7(t)     =  [(2  +  e)  -  t   (1  +  ^)]   e~*  u(t) 

(2+9)   s  +  1 


Step  7.         Z(g)   SB 


(5  +  1)2 


(113) 


(llh) 

(115) 
(116) 

(117) 
(118) 


Step  0.     Tlie  ^Tithcsis  of  the  f:'.ltGr  as  a  driving  point  inpedance  is 
sliavm.  in  Fisiu'c  10» 


I  in 


farad 

2  4-  e 


< 

S     3  +  26 


(2  +  «)•',  ,, 

henry     {^ 

(1  +  9)2  p 


ohn 


2+9 
1  +  w 


olim 


E  out 


Figure  10,     Predictor  Tor  a  Jlrst  Order  Po3;,Tionial 
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Tables  1  and  2  give  the  inverse  mtrices  for  polynoiaials  up  to  and 
including  tiie  fifth  order.     Table  1  is  for  roctang^ar  ^Tcich'tinc  vrLth  a 
snoo tiling  tij:«  of  1  second  vrhile  Table  2  is  for  exj^onential  vfeighting 
"With  a  soootldng  tine  constant  of  1  second.     If  T  is  otlicr  tlian  1  second, 
the  tables  r-iay  still  be  used  by  scaling  tlie  problen  proporily.     Por 
exai:ple,  the  expression  for  the  v/eighting  function  of  a  predictor  for  a 
first  Older  pol;>Tioinial  using  rectangiilar  vreighting  and  a  sinoothirg  time 
of  1  second  is 


W(t) 


h  4-  6  e    -     (6  *  12  e)  t      u(t)    -    u(t  -  1)  (119) 


and  the  corresponding  ecqjression  for  a  general  smoothing  tine  T  is 

(120) 


W(t,  T)   =    i 


U^6-    -     (6^12-)i 
T  T       T 


u(t)     -    u(t  -  T) 


Figures  11  and  12  shoiT  the  fanily  of  polynonial  s:.ioothing  filters 
derived  from  Table  2.     Tliese  filters  are  of  the  low  pass  type  rdth  an 
increasing  resi^onse  to  liigher  frequencies  as  the  order  of  the  pollynonial 
is  increased.     The  corrplex  frequency  transfer  functions  of  the  n  th 
order  polynomial  sraoothing  filters  shovm  in  Figures  11  and  12  have  been 
found  to  talce  the  form 

H(s)    =    1 ^ (121) 

(s  +  1)^  ♦  1 

Figure  13  shovfs  the  anplitude  and  phase  response  versus  frequency 
for  first  order  poIl;>Tioinial  predictors  sinLlar  to  the  one  in  Figure  10« 
Figure  Ih  shotrs  the  saxie  infors'iation  about  tlie  polynonial  filters  in 
Figures  11  and  12.  A  positive  increase  in  prediction  tine  or  polynonial 
order  evidently  tends  to  increase  the  noise  output  of  the  filter. 
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Table  1. 
Inverse  or     Cj-^     liitrices  for  Po3ynonials  of  Order  itu 
Ll(t)    =    u(-t)     -    u(--U-l) 


n    —    0 


[•] 


111 


h         6 
6        12 


n    =    2 


9 

36 

30 

36 

192 

180 

30 

180 

180 

n    =  3 


16 

120 

2l;0 

lliO 

120 

1200 

2700 

1680 

2liO 

2700 

6U0O 

h200 

lUo 

1600 

U200 

2800 

n   =    U 


2$ 

300 

10^0 

iloo 

630 

300 

1800 

18900 

26880 

12600 

10:^0 

18900 

79380 

117600 

56700 

lUoo 

26880 

117600 

179200 

88200 

630 

12600 

56700 

88200 

iiinoo 

n    =   5 


36 

630 

3360 

7560 

7560 

2772 

630 

lli700 

88200 

?11680 

220500 

83160 

3360 

88200 

5610^80 

11:11200 

1512000 

582120 

7560 

211680 

l)iTT200 

3628800 

3969000 

1552320 

7560 

220500 

1512000 

3969000 

)i)'10000 

I7U636O 

2772 

83160 

582120 

1552320 

I7U636O 

6985U4 

U3 
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Table  1. 
Inverse  or  Cj;-^  Ifatrices  for  PoHynonials  of  Order  nu 

U(t)   =   u(-t)   -   u(--tr-l) 


m  —  0 


['] 


m 


h  6 

6   12 


n  =  2 


9 

36 

30 

36 

192 

180 

30 

180 

180 

n  =  3 


16 

120 

2liO 

lliO 

120 

1200 

2700 

1680 

2liO 

2700 

6U80 

1:200 

lliO 

1680 

U200 

2800 

n  =  U 


n  =  ^ 


2S 

300 

10^0 

lUOO 

630 

300 

h800 

18900 

26880 

12600 

10^0 

18900 

79380 

117600 

^6700 

lUoo 

26880 

117600 

179200 

88200 

630 

12600 

^6700 

88200 

I'inoo 

36 

630 

3360 

7^60 
7^60 
2772 


630 

ll;700 

88200 

211680 

220^00 

83160 


3360 

88200 

5610:80 

1101200 

1512000 

582120 


7560 
211680 
1111200 
3628800 
3969000 
1552320 


7560 

220500 

1512000 

3969000 

hiaoooo 
17U6360 


2772 

83160 

582120 

1552320 

I7U636O 

6965hJ4 


li3 


i 


I 
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Table  2. 
Inverse  or  jCj-jl   Lfatrices  for  Polynomials  of  Order  m. 
U(t)   =    e*  u(-t) 


n    =    0 


H 


m 


1 
1 


n    =    2 


3  3  1/2 

3         S  1 

1/2  1  lA 


in    =  3 


li 

6 

2 

V6 


6 

Ih 

11/2 

1/2 


2 

1V2 

5/2 

1/U 


1/6 
1/2 

vu 

1/36 


n   =.   li 


5 

10 

5 

1/2U 


10 

30 

3^2 

19/6 

1/6 


5 
35/2 
23/2 

9A 

1/8 


5/6 

19/6 

9A 

17/36 

1/36 


V2l^ 
V6 

1/8 

1/36 

1/576 


m   =   5 


6 

15 

10 

15/6 
i/U 

1/120 


15 
^$ 

85/2 

23/2 

29/21; 

l/2lt 


10 

85/2 

73/2 

127/12 

7/6 

V2lt 


15/6 

23/2 

127/12 

13A 

3/8 

1/72 


29/21 
7/6 

3/8 

13/2C8 

1/576 


1/120 
1/2U 
l/2li 
1/72 

1/576 

1/iUiOO 


10; 


1    =^      >  1 


0*^  =b 


m    =   0 


m   =  1 


0.333-r 


0.25  =T= 


3.2 


1,6 


•1,6 


1.3'62    -r- 


m    =    2  m   =   3 

FlgTjre  11.     Zero  Through  Third  Order  Poljmoinial  SruDothinc  lifters. 

M(t)    =    e*  u(-t) 

All  resistances,  inductances,  and  capacitances  are  given  in  ohms,  henries, 

and  farads  resi'ectively.  * 


hS 


0.2: 


2.5 


1.25 


m   =    U 


1,C<>7 


l,lli3  ^ 


1.75 


3.^61 


0.5U7 


0.1667  =r= 


I? 


2.057 


1.029 


•1.711 


0,91l5- 


m    =    5 


5.689 


?  2.8II1 


2.19h 
2.215 


^ 


16.25 


'O.903 


127 


1. 


Figure  12.     Pom-th  and  51fth  Order  PolynoniLal  Snootliirg  Filters. 
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6.  Filters  for  a  Combination  of  Functions. 

Since  any   additive  or  nultiplicative  combination  of  simisoidsj 
Gxponentialsj  and  poU^Tiomials  is  a  solution  to  a  reduced  equation  of 
some  linear  differential  equation  Tdth  constant  coefficients,  it  may- 
be handled.  Such  functions  as  t  sin  t,  e  sin  t,  and  t  e  sin  t  nrsy 
tlius  be  treated  after  the  conplcte  sot  of  fimctions  forrdnc  ^   closed 
set  upon  translation  in  tine  is  found.  A  filter  which  is  tlie  additive 
coribination  of  a  first  order  polynomial  and  a  sinusoid  can  be  designed 
as  foUovrs. 


Step  !•         g3_  ==  cos  t 


g2  ^  sin  t 


Step  3. 


Step  U. 


23 


gu 


=  1 


=  t 


Step  2.         M(t)  =  e**^  u(-t) 


H  - 


H- 


3/'. 

-V^ 

1/2 

•V^ 

2/^ 

-V2 

V2 

-V2 

1 

0 

V2 

-1 

12 

-h        . 

-Hi        -6 

-U 

8 

8          2 

•lU 

8 

20          8 

-6 

2 

6          I 

0 

V2 

-1 
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(122) 
(123) 

(12li) 
(125) 
(126) 


(127) 


(128) 


h9 


i 


step  5.    gi(0)  =  1 


g2(0)  =0 


g3(0)  =1 


gl;(0)    =0 

3t6p  6* 

W(t)    =    6-2t-2co3t- 

Step  7. 

0(3)  _       Us^  +  6s^  4  6s  4  2 

U  sin  tj  e"^  u(t) 


s^  ^  lis^  +  73^  4  6s  -f  2 


(129) 

(130) 

(131) 

(132) 

(133) 
(13U) 


Step  8.  Figure  1^  illustrates  one  method  of  real  i  zinc  this  filter  as 
a  driving  point  ir^edance* 


I  in 


16/7  henry 


8/7  ohn 


VU  farad  =: 


8/^  ohn 


E  out 


h9/6l4  farad  -r- 


>  256/35  heniy 
128/h9  ohn 
128/35  ohn 


Figure  15  •  &ioothing  Filter  for  a  Ramp  and  Sine  Wave* 
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Ftom  equation  (13U) 

Z(dl)  =  1  (135) 

2(0)  --  1  (136) 

Tlius  in  the  steady  state  the  drivinc  point  iispedance  for  a  sinusoidal 
input  Ydth  an  angular  fj?equency  of  one  radian  per  second  or  for  a 
constant  input  is  a  pure  resistance  of  one  olm.  Vflien  tlie  input  is 
equal  to  t,  the  output  oay  be  found  by  rnultiplyinc  Z(s)  try-  the  Laplace 
transform  of  t,  tliat  is  l/s-«  The  inverse  Laplace  transform  of  the 
product  has  a  steady  state  tern  equal  to  t.  The  stGactr  state  output 
divided  "by  the  input  is  tlien  t/t  or  1.  Although  the  filter  is  con- 
structed ;Titli  reactive  elements,  its  steadj^'  state  irpedancc  to  a 
sinusoid,  a  constant,  and  a  raznp   input  appears  to  be  a  pure  resistance 
of  one  oha.  A  phenomenon  akin  to  resonance  is  tlTus  observed  for 
fiinctions  of  tine  other  than  sine  Tmves. 

7#  Response  of  Various  Polynoiaial  Filters  in  the  Absence  of  lloise. 

nil  order  to  evaluate  the  polynomial  filters,  the  response  of  the 
filters  to  standard  input  signals  in  the  absence  of  noise  can  be 
calculated  from  equation  (5).  Tlie  standard  signals  chosen  for  this 
evaluation  are 

Rj_  =  1  (a  constant)  (137) 

Rg  =  1  +  t  (a  ramp)  (I38) 

R.^  =  1  4.  t  ^  t^  (a  quadratic)  (139) 


The  graphs  on  the  folloTrinc  paces  siiow  tiio  desired  response  to  these 
signals  as  dashed  lines  and  the  actusil  response  as  solid  lines.  Ibr 
smoothing  filters  the  desired  steacfy  state  responses  to  R^,  Ro*  and  YU 
are  1,  1ft,  aiid  1  ♦■  t  4-  t^;  for  integrating  filters,  t,  t  f  t2/2, 
and  t  4-   t2/2  <♦•  i^/3j   and  fbr  differentiators,  0,  1,  and  1  +  2t  respec- 
tively. 

Jlgures  16  through  21  inclusive  illustrate  tlie  response  of  srcothing 
filters  designed  for  constants,  rajqjs,  and  quadratics  with  both  rectan- 
gular and  exponential  vreighting.  It  should  be  noted  that  the  cteac^ 
state  response  of  tlie  rectangular  filter  is  reached  abruptly  at  the  end 
of  the  smootliing  period  Tdiile  the  eixponential  filter  requires  five  to 
ten  time  constants.  K-ltors  designed  to  smooth  an  ra  th  order  polynonLal. 
•will  also  smooth  lower  order  polynomials.  If  a  polynomial  of  order 
m  ♦•  q  f  1  is  put  into  the  filter,  the  output  vrill  have  an  error  -vrhich  is 
a  polynomial  in  t  of  order  q,  Ydiere  q  is  a  positive  constant. 

The  response  of  polynomial  smoothing  filters  to  a  unit  ctep  input  is 
illustrated  in  Figures  22  and  23*  As  the  order  of  the  polynomial  for 
"uhich  the  filter  is  designed  increases,  the  overshoot  and  frequency  of 
transient  oscillation  is  observed  to  increase. 

FigTires  2U,  2^,  and  26  sho>T  the  response  of  filters  designed  to 
predict  a  ran^J  function  vrlth  four  different  value g  of  prediction  tine 
and  three  different  error  memory  functions.  Tlie  influence  of  the  error 
memory  function  upon  tlie  transient  response  can  be  clearly  seen. 

Beeponses  of  filters  dei^Lved  by  dividing  the  conr>lex  frecjuency  trans- 
fer functions,  H(s),  for  the  constant  and  ranp  smootliing  filters  by  the 
variable  s  are  shorm  in  Figures  27  through  30.  These  curves  indicate  that 
a  filter  -vriiich  is  to  integrate  polynomials  up  to  the  m  th  order  should  be 
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based  upon  poUynoioial  snxDotliiiic  filters  of  tlie  n  4-  p  th  order,  v.-herc  p  is 
the  nunber  of  integrations  to  be  i-K^rfomed.     Thic  niicht  be  anticipated 
fron  tlic  fact  tiiat  an  intccrator  increases  the  order  of  t!ic  output  poly^- 
nondal  as  coripared  Tiith  the  input  polynoinial. 

Differentiators  liave  respoiises  as  illuo-trated  by-  Ficuros  31  tlirough 
36.     These  filters  differentiate  all  poVnoiiLals  up  to  tlie  order  n  if  they 
are  based  on  polynonial  snootliinc  filters  of  at  least  tlie  order  m  -  q, 
TThcre  q  is  the  n\inbcr  of  differentiations.     If  the  rectangular  or  c:q^or>- 
ential  error  raemor^'-  functions  are  uced,   the  res^^onse  curves  shoiv  discon- 
timiities  in  the  output.     These  discontinuities  indic:;tc  that  the  filters 
have  an  infinite  frequenc;/'  resi:)02ise  and  hence  -aould  not  be  very  practical 
for  eliminating  liich  flrequency  noise  coraoonents.     Tiie  discontinuities 
result  fron  the  fact  tiiat  the  derivative  of  a  unit  step  function  is  the 
unit  iEB5ulse  or  Dii*ac  delta  fimction  and  mc^  be  eliminated  by  choosinc 
error  menoiy  functions  which  never  liave  an  infinite  slope.     The  responses 
of  tvro  filters  having  such  error  rnemory  functions  are  shoivn  in  Figures  35 
and  3<^« 
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